{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 泰坦尼克乘客生存预测\n",
    "\n",
    "1. 准备阶段： 对训练集，测试集的数据进行探索， 分析数据质量，并对数据进行清洗，然后通过特征选择对数据进行降维，方便后续分类运算\n",
    "2. 分类节点： 首先通过训练集的特征矩阵， 分类结果得到决策树分类器， 然后对分类器应用于测试集； 对决策树分类器的准确性进行分析， 并对决策树模型进行可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 数据探索\n",
    "数据探索这部分虽然对分类器没有实质作用，但是不可忽略。我们只有足够了解这些数据的特性，才能帮助我们做数据清洗、特征选择。\n",
    "那么如何进行数据探索呢？这里有一些函数你需要了解：\n",
    "* 使用info()了解数据表的基本情况：行数、列数、每列的数据类型、数据完整度；\n",
    "* 使用describe()了解数据表的统计情况：总数、平均值、标准差、最小值、最大值等；\n",
    "* 使用describe(include=[‘O’])查看字符串类型（非数字）的整体情况；\n",
    "* 使用head查看前几行数据（默认是前5行）；\n",
    "* 使用tail查看后几行数据（默认是最后5行）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.7+ KB\n",
      "None\n",
      "______________________________\n",
      "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
      "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
      "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
      "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
      "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
      "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
      "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
      "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
      "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
      "\n",
      "            Parch        Fare  \n",
      "count  891.000000  891.000000  \n",
      "mean     0.381594   32.204208  \n",
      "std      0.806057   49.693429  \n",
      "min      0.000000    0.000000  \n",
      "25%      0.000000    7.910400  \n",
      "50%      0.000000   14.454200  \n",
      "75%      0.000000   31.000000  \n",
      "max      6.000000  512.329200  \n",
      "______________________________\n",
      "                       Name   Sex  Ticket Cabin Embarked\n",
      "count                   891   891     891   204      889\n",
      "unique                  891     2     681   147        3\n",
      "top     Jermyn, Miss. Annie  male  347082    G6        S\n",
      "freq                      1   577       7     4      644\n",
      "______________________________\n",
      "   PassengerId  Survived  Pclass  \\\n",
      "0            1         0       3   \n",
      "1            2         1       1   \n",
      "2            3         1       3   \n",
      "3            4         1       1   \n",
      "4            5         0       3   \n",
      "\n",
      "                                                Name     Sex   Age  SibSp  \\\n",
      "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
      "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
      "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
      "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
      "4                           Allen, Mr. William Henry    male  35.0      0   \n",
      "\n",
      "   Parch            Ticket     Fare Cabin Embarked  \n",
      "0      0         A/5 21171   7.2500   NaN        S  \n",
      "1      0          PC 17599  71.2833   C85        C  \n",
      "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
      "3      0            113803  53.1000  C123        S  \n",
      "4      0            373450   8.0500   NaN        S  \n",
      "______________________________\n",
      "     PassengerId  Survived  Pclass                                      Name  \\\n",
      "886          887         0       2                     Montvila, Rev. Juozas   \n",
      "887          888         1       1              Graham, Miss. Margaret Edith   \n",
      "888          889         0       3  Johnston, Miss. Catherine Helen \"Carrie\"   \n",
      "889          890         1       1                     Behr, Mr. Karl Howell   \n",
      "890          891         0       3                       Dooley, Mr. Patrick   \n",
      "\n",
      "        Sex   Age  SibSp  Parch      Ticket   Fare Cabin Embarked  \n",
      "886    male  27.0      0      0      211536  13.00   NaN        S  \n",
      "887  female  19.0      0      0      112053  30.00   B42        S  \n",
      "888  female   NaN      1      2  W./C. 6607  23.45   NaN        S  \n",
      "889    male  26.0      0      0      111369  30.00  C148        C  \n",
      "890    male  32.0      0      0      370376   7.75   NaN        Q  \n",
      "______________________________\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# 数据加载\n",
    "train_data = pd.read_csv('./train.csv')\n",
    "test_data = pd.read_csv('./test.csv')\n",
    "\n",
    "# 数据探索\n",
    "\n",
    "print(train_data.info())\n",
    "print('_'*30)\n",
    "print(train_data.describe())\n",
    "print('_'*30)\n",
    "print(train_data.describe(include=['O']))\n",
    "print('_'*30)\n",
    "print(train_data.head())\n",
    "print('_'*30)\n",
    "print(train_data.tail())\n",
    "print('_'*30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PassengerId      0\n",
      "Survived         0\n",
      "Pclass           0\n",
      "Name             0\n",
      "Sex              0\n",
      "Age            177\n",
      "SibSp            0\n",
      "Parch            0\n",
      "Ticket           0\n",
      "Fare             0\n",
      "Cabin          687\n",
      "Embarked         2\n",
      "dtype: int64\n",
      "______________________________\n",
      "PassengerId      0\n",
      "Pclass           0\n",
      "Name             0\n",
      "Sex              0\n",
      "Age             86\n",
      "SibSp            0\n",
      "Parch            0\n",
      "Ticket           0\n",
      "Fare             1\n",
      "Cabin          327\n",
      "Embarked         0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(pd.isna(train_data).sum())\n",
    "print(\"_\"*30)\n",
    "print(pd.isna(test_data).sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 数据清洗\n",
    " 通过数据探索， 我们发现Age, Fare和Cabin这三个字段的数据有缺失；\n",
    " - Age为年龄字段， 是数值型， 可以通过平均值进行补齐；\n",
    " - Fare为船票价格， 是数值型， 也可以通过平均值来补齐\n",
    " - Cabin为船舱，有大量的缺失值。在训练集和测试集中的缺失率分别为77%和78%，无法补齐\n",
    " > 查看缺失值的个数 \n",
    " ```python\n",
    " print(pd.isna(train_data).sum())\n",
    " print(pd.isna(test_data).sum())\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 年龄使用平均值填充\n",
    "train_data['Age'].fillna(train_data['Age'].mean(), inplace=True)\n",
    "test_data['Age'].fillna(train_data['Age'].mean(), inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用票价平均值来填充\n",
    "train_data['Fare'].fillna(train_data['Fare'].mean(), inplace=True)\n",
    "test_data['Fare'].fillna(test_data['Fare'].mean(), inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "S    644\n",
      "C    168\n",
      "Q     77\n",
      "Name: Embarked, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(train_data['Embarked'].value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 发现一共就3个登陆港口， 其中S港口的人数最多， 占到了72%， 因此我们将其余缺失的Embarked的数值均设为S\n",
    "# 使用登录最多的港口来挑冲登录的港口的NaN值\n",
    "train_data['Embarked'].fillna('S', inplace=True)\n",
    "test_data['Embarked'].fillna('S', inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 特征选择\n",
    "\n",
    "特征选择是分类器的关键， 特征选择不同， 得到的分类器也不同。\n",
    "\n",
    "> 数据探索阶段，可以发现编号，Ticket字段为船票号码，杂乱无章且无规律，所以对于一个顾客能否生存下来造成太大影响"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 特征选择\n",
    "features = ['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked']\n",
    "train_features = train_data[features]\n",
    "train_labels = train_data['Survived']\n",
    "test_features = test_data[features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction import DictVectorizer\n",
    "dvec = DictVectorizer(sparse=False)\n",
    "train_features = dvec.fit_transform(train_features.to_dict(orient='record'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Age', 'Embarked=C', 'Embarked=Q', 'Embarked=S', 'Fare', 'Parch', 'Pclass', 'Sex=female', 'Sex=male', 'SibSp']\n"
     ]
    }
   ],
   "source": [
    "print(dvec.feature_names_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[22.          0.          0.         ...  0.          1.\n",
      "   1.        ]\n",
      " [38.          1.          0.         ...  1.          0.\n",
      "   1.        ]\n",
      " [26.          0.          0.         ...  1.          0.\n",
      "   0.        ]\n",
      " ...\n",
      " [29.69911765  0.          0.         ...  1.          0.\n",
      "   1.        ]\n",
      " [26.          1.          0.         ...  0.          1.\n",
      "   0.        ]\n",
      " [32.          0.          1.         ...  0.          1.\n",
      "   0.        ]]\n"
     ]
    }
   ],
   "source": [
    "print(train_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 决策树模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,\n",
       "                       max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort=False,\n",
       "                       random_state=None, splitter='best')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# 构造ID3决策树\n",
    "clf = DecisionTreeClassifier(criterion='entropy')\n",
    "\n",
    "# 决策树训练\n",
    "clf.fit(train_features, train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 模型预测&评估\n",
    "\n",
    "在预测中， 我们首先需要得到测试集的特征值矩阵， 然后使用训练好的决策树clf进行预测， 得到预测结果pred_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_features = dvec.transform(test_features.to_dict(orient='record'))\n",
    "# 决策树预测\n",
    "pred_labels = clf.predict(test_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 0 0 1 1 0 0 0 0 0 0 0 1 0 1 1 0 1 1 0 1 1 1 0 1 0 1 1 1 0 0 0 1 0 1 0 0\n",
      " 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 1 0 1 0 0 0 0 1 0 0 0 1 1 1 1 0 0 1 1 0 0 0\n",
      " 1 0 0 0 0 1 1 0 0 0 1 0 1 0 1 1 0 0 0 0 0 0 1 1 1 1 1 0 0 0 1 0 0 0 0 0 0\n",
      " 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0\n",
      " 0 0 1 0 0 1 0 0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 0 0 0 0 0 1 1 1 1 1 0 0 1 0 1\n",
      " 0 1 0 0 0 0 0 1 1 1 0 1 0 0 0 1 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1 0 1 0\n",
      " 1 1 1 0 0 1 0 0 0 1 1 0 1 0 1 1 1 1 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 0 0 0 1\n",
      " 0 0 0 1 1 0 0 0 0 0 0 1 0 1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 1 0 0 0 1 1 0 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 0 0 0 1 0 0\n",
      " 1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 0 0 1 0 1 1 1 1 0 0 1 1 1\n",
      " 0 1 0 0 1 1 0 0 0 0 0 0 0 1 1 1 0 0 1 0 1 1 0 0 0 1 0 1 0 0 1 0 1 0 0 1 0\n",
      " 0 1 0 1 1 0 0 1 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "print(pred_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "score准确率为0.9820\n"
     ]
    }
   ],
   "source": [
    "## 得到决策树准确率\n",
    "acc_decision_tree = round(clf.score(train_features, train_labels), 6)\n",
    "print(u'score准确率为%.4lf'% acc_decision_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross_val_score准确率0.7835\n"
     ]
    }
   ],
   "source": [
    "# 使用K折交叉验证\n",
    "\n",
    "import numpy as np\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "print(u'cross_val_score准确率%.4lf' % np.mean(cross_val_score(clf, train_features, train_labels, cv=10)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
